With increasing demand for more information to be supplied to homes and/or businesses, network communication providers have been continuously upgrading and expanding their networks' performance as well as capacity. When a network, for instance, reaches its service capacity, additional network equipments are typically added to improve overall network capacity to meet such demand. For example, a communications network can add network equipment such as routers and switches to keep up with the demand. A problem associated with a growing network, however, is to keep up with statistics collection from all nodes and/or network elements (“NEs”) whether they are existing NEs or newly added NEs in the network.
Traditionally, statistics collection over a communications network involves downloading statistics files from NEs, processing the statistics files, and producing output files on a defined time period. For example, output files are typically generated based on collected statistics files and are available for review within the time period such as every 15 minutes. When a network grows, new NEs or nodes are added and consequently, the statistics from the newly added NEs and nodes need to be collected and processed. Accordingly, the load on the statistics data collector increases when the network grows. As the network grows, the scalability and performance issues cause statistics collection to lag behind over a large network. As such, a delay in availability of statistics data can occur if a network grows.
Since large amount of statistics data is processed and stored on a regular interval, a conventional approach for handling such massive amount of data is to store the data in a file format such as a comma separated value (“CSV”) file. If a report is desirable, the report will be generated from, for example, CSV files.